Section: New Results

Augmented Environments

A large part of the research on augmented environments specializes in the use of visual media. In WAM, we focus on the use of audio media and we put a strong emphasis on mobility.

We have developed the first indoor augmented reality audio navigation application running on personal AR devices such as mobile phones. The main idea behind the development of this application was a joint use of three concepts:

• Continuous localization by using embedded sensors together with physiological models of walking and assumptions about walking in structured indoor environments.

• OpenStreetMap Indoor Mapping used for map-aided positioning, assistive routing for visually impaired people, and environmental queries through audio panoramics.

• Guidance and navigation through AR audio, both 3D and environmental, with mixing of synthetic and natural sounds and support for timely audio information presentation.

We have demonstrated that these concepts are inter-dependent [12] , and that bringing them together is a way to find new solutions to problems which are difficult to tackle when looking at them separately. These three concepts are implemented with web technologies we use XML languages and XML tools for interactive audio, building modeling, and personal navigation module configuration. This enables easy authoring of sound objects or audio icons used for building sonification [15] , indoor navigation maps and panoramics, and walking models. Adaptability of navigation to preferences of people is based on the concept of audio stylesheets for OpenStreetMap data rendering, XML configuration of the Pedestrian Dead Reckoning module, and assistive routing specification.

We have developed two mobile browsers and a framework for generic navigation:

1. The Mixed Reality Browser [10] that we have developed can display PoI content either remotely through panoramics with spatialized audio, or on-site by walking to the corresponding place. MRB is the only browser of geolocalized data to use a declarative XML format for PoIs, panoramics, 3D audio and to be based on HTML5 both for the iconic and full information content of PoIs. MRB can be used for any kind of augmented reality visits. A cultural heritage visit of Grenoble (see section 8.1.2 ) has been realized with the tourist office of Grenoble and the CCSTI (Centre de Culture Scientifique Technique et Industrielle de Grenoble).

2. The Pedestrian Way Browser that we have developed can be used for indoor-outdoor navigation with assistive audio technology for visually impaired people on pedestrian ways with precise geospatial description. Its main characteristic is to be based entirely on the OpenStreetMap XML format for the representation of the route. We anticipate that in the context of the european project Venturi (see section 8.3.1 ), we will have a convergence of the MRB and PWB, allowing visually impaired people to undertake cultural heritage visit. An demonstration showing the use of the PWB in a structured outdoor environment is available online: http://www.youtube.com/watch?v=h2b8yfCauZ8

3. We have created an extensible client-server framework named TARA which allows navigation on an OpenStreetMap XML graph (indoor and outdoor) by computing routes in real-time. User preferences, like stairs versus lift, are supported through a ponderation of paths in the routing algorithm. The client is an HTML5 running in the browser on mobiles and desktop computers. It can therefore be used for simulation, to test or learn a route before the navigation in the real world. The user interface is based on three modalities, touch, audio and visual and can be operated by visually impaired people through VoiceOver using only touch and audio. Localization through embedded or external sensors is not mandatory as step by step instructions can be accessed through touch modality. The server is a full REST server (Sling-Apache) giving priority to the representation of geospatial resources and allowing environmental queries through the use of XQuery.